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In this paper, with the notion of independent identically distributed 
random variables under sub-linear expectations initiated by Peng, we 
derive three kinds of strong laws of large numbers for capacities. More- 
over, these theorems are natural and fairly neat extensions of the classi- 
cal Kolmogorov's strong law of large numbers to the case where proba- 
bility measures are no longer additive. Finally, an important feature of 
these strong laws of large numbers is to provide a frequentist perspective 
on capacities. 
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Introduction 



^ , The classical strong laws of large numbers (strong LLN) as fundamental limit theorems 

in probability theory play an important role in the development of probability theory 
and its applications. The key in the proofs of these limit theorems is the additivity 
of the probabilities and the expectations. However, such additivity assumption is not 
reasonable in many areas of applications because many uncertain phenomena can not 
be well modeled using additive probabilities or additive expectations. More specifically, 
motivated by some problems in mathematical economics, statistics, quantum mechanics 
and finance, a number of papers have used non-additive probabilities (called capacities) 
and nonlinear expectations (for example Choquet integral/expectation, ^-expectation) to 
describe and interpret the phenomena (see for example, Chen and Epstein [1], Feynman 
[4], Gilboa [5], Huber [16], Peng [8,9], Schmeidler [18], Wakker [19], Walley and Fine 
[20], Wasserman and Kadane [21]). Recently, motivated by the risk measures, super- 
hedge pricing and modelling uncertainty in finance, Peng [11,12,13,14,15] initiated the 
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notion of independent and identically distributed (HD) random variables under sub-linear 
expectations. Under this framework, he proved the weak law of large numbers and the 
central limit theorems (CLT). However, Peng's techniques can not be extended to prove 
the strong laws of large numbers. In this paper, we develop new approaches to solving this 
problem. We obtain three strong laws of large numbers for capacities in this framework. 
All of them are natural and fairly neat extensions of the classical Kolmogorov's strong law 
of large numbers, but the proofs here are different from the original proofs of the classical 
strong law of large numbers. 

Now we describe the problem in more detail. For a given set V of multiple prior 
probability measures on J 7 ), we define a pair (V,v) of capacities by 

Y(A) := sup P(A), v(A) := inf P(A), VA G 7. 
p^-p PeP 

The corresponding Choquet integrals/expecations (Cy,C v ) are defined by 

poo i>0 

C V [X] := / V(X > t)dt + / [V(X > t) - l]dt 

JO J-oo 

where V is replaced by v and V respectively. 

The pair of so-called maximum-minimum expectations (E, £) are defined by 

E[£]:=sup£ P [£], £[i]:=miE P [i]. 

P£P P€V 

Here and in the sequel, Ep denotes the classical expectation under probability P. 

In general, the relation between Choquet integral and maximum -minimum expecta- 
tions is as follows: For any random variable X, 

E[X] < C Y [X], C V [X\ < S[X\. 

Note that under some very special assumptions on V and V, both inequalities could 
become equalities (see for example Gilboa [5], Huber [16], Schmeidler [18]). 

Given a sequence {X, i }°^ =1 of IID random variables for capacities, the earlier papers 
related to strong laws of large numbers are Dow and Werlang [2] and Walley and Fine 
[20]. However, the more general results for strong laws of large numbers for capacities 
were given by Marinacci [6,7] and Epstein and Schneider [3]. They show that, on full set, 
any cluster point of empirical averages lies between the lower Choquet integral C„[Xi] 
and the upper Choquet integral CV[-Xi] with probability one under capacity v. That is 

v ( u E VI : CAXA < liminf- VXi(w) < limsup - V X { (u ) < CylxA = 1. 
\ i=i i=i / 

Marinacci [6,7] obtains his result under the assumptions that V is a totally monotone 
capacity on a Polish space Q, random variables {X n } n >i are bounded or continuous. 
Epstein and Schneider [3] also show the same result under the assumptions that V is 
rectangular and the set V is finite. 
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Since the gap between the Choquet integrals C^fX] and Cy[X] is bigger than that 
of the maximum-minimum expectations S[X\ and E[X] for all X, it is of interest to ask 
whether we can obtain a more precise result if the Choquet integrals/expctations in the 
above equality are replaced by maximum-minimum expectations. That is 



/ n n \ 

\u E tt : EVKA < liminf -VXjfw) < limsup - ^ XAu) < MXA 
\ n^oo ,woc n ^ ) 

\ i=\ t=i / 



1. 



The first result in this paper is to show that the above equality is still true in Peng's 
framework even under some weaker assumptions. Furthermore, motivated by this result, 
we establish two new laws of large numbers. The first is to show that there exist two 
cluster points of empirical averages which reach the minimum expectation and the 

maximum expectation E[Xl] respectively under capacity V. That is 



V 



( 1 n 

iu e n : limsup -J]X 4 (w) = E[X X 

\ t=l 
1 n 

cueQ: liminf - XAou) = E\X X 



The second is to prove that the cluster set of empirical averages coincides with the interval 
between minimum expectation S[X{\ and maximum expectation E[Xl]. That is, if C({x n }) 
is the cluster set of {x n }, then 



V \u) e n : C 



^E^njj = [£[*!], =1- 



Obviously, if capacity V or v in the above results is a probability measure, all of our main 
results are natural and fairly neat extension of the classical Kolmogorov's strong law of 
large numbers. Moreover, an important feature of our strong laws of large numbers is to 
provide a frequentist perspective on capacities. 

Finally, our results also imply that E[XJ] is the smallest interval of all intervals 

in which the limit point of empirical averages lies with probability (capacity v) one. 



2 Notation and Lemmas 

In this section, we will recall briefly the notion of IID random variables and sub-linear 
expectations initiated by Peng. 

Let J 7 ) be a measurable space, and H be a set of random variables on (Q, J 7 ). 



DEFINITION 1 A functional E on T-L h- > (— oo, +oo) is called a sub-linear expectation, 
if it satisfies the following properties: for all X, Y G "H, 
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(a) Monotonicity: X > Y implies E[X] > E[Y]. 

(b) Constant preserving: E[c] = c, Vc G M. 

(c) Sub-additivity: E[X + Y}< E[X] + E[Y]. 

(d) Positive homogeneity: E[XX] = AE[X],VA > 0. 

Given a sub-linear expectation E, let us denote the conjugate expectation £ of sub- 
linear E by 

E[X\ := -E[-X], VX G H. 

Obviously, for all X G H, £[X] < E[X\. 

Furthermore, let us denote a pair (V, v) of capacities by 

V(A):=E[I A ], v(A):=€[I A ], VA G T. 

It is easy to check that 

V(A) +v(A c ) = 1, ViGJ, 
where A c is the complement set of A. 

DEFINITION 2 A set function V : J 7 — > [0, 1] is called a continuous capacity if it 
satisfies 

(1) v(<p) = o,v(n) = i. 

(2) V(A) < V(B), whenever A C B and A, B G J= . 

(3) V(A n )tV(A),ifA n tA 

(4) V(A n ) I V(A), ifA n I A, where A n ,Ae T. 

Assumption 1 Throughout this paper, we assume that (V, v) is a pair of continuous 
capacities generated by sub-linear expectation E and its conjugate expectation S. 

The following example shows that the maximum expectation generated by multiple 
prior probability measures is a sub-linear expectation and the associated capacities are 
continuous (see for example, Huber and Strassen [17]). 

Example i Let V be a weakly compact set of probability measures defined on (fl, J 7 ). 
For any random variable £ 7 we denote upper expectation by 

EK]=SU P £gK]. 
QeV 

Then E[-] is a sub-linear expectation. Moreover, the capacities V and v given by 

Y(A) = sup Q(A), v(A) = inf Q(A), VA G J= 
Qev Q^ v 

are continuous. 
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The following is the notion of IID random variables under sub-linear expectations 
introduced by Peng. 

Definition 3 (Peng [11]) 

Independence: Suppose that Yi, Y 2 , ■ ■ ■ , Y n is a sequence of random variables such that 
Yj G T-L. Random variable Y n is said to be independent of X := (Yi, • • • , Y n _i) under 
E, if for each measurable function <p on R n with (p(X, Y n ) G H and ip(x,Y n ) G % 
for each x G M™ _1 , we /lave 

E[^(X,y n )]=E[^(X)], 
w/iere Tp(x) := E[<^(x, F n )] and Tp(X) G "H. 

Identical distribution: Random variables X and Y are said to be identically dis- 
tributed, denoted by X = Y , if for each if such that ip(X), ip(Y) G T-L, 

E[<p(x)]=E[<p(y)]. 

IID random variables: A sequence of random variables {Xj}^ is said to be IID, if 
Xi = X 1 and X i+1 is independent of Y := (Xi, ■ ■ ■ , XA for each % > 1. 

The following lemma shows the relation between Peng's independence and pairwise 
independence in Marinacci [6,7]. 

L/EMMA 1 Suppose that X,Y G 7i are two random variables. E is a sub-linear ex- 
pectation and (V, v) is the pair of capacities generated by E. If random variable X is 
independent of Y under E, then X is also pairwise independent of Y under capacities V 
and v, i.e. for all subsets D and GcK, 

V(X G D, Y G G) = V(X G D)V{Y G G) 

holds for both capacity V and v. 

PROOF. If we choose (p(x,y) = Id(x)Ig(v) for E, by the definition of Peng's inde- 
pendence, it is easy to obtain 

Y(X eD,Y eG) = Y(X G D)Y(Y G G). 

Similarly, if we choose ip(x,y) = —Id{x)Ig{]j) for E, it is easy to obtain 

v(X G D,Y G G) =v(X G D)v{Y G G). 

The proof is complete. 

Borel-Cantelli Lemma is still true for capacity under some assumptions. 
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L/EMMA 2 Let {A n , n > 1} be a sequence of events in T and (V, v ) be a pair of capacities 
generated by sub-linear expectation E. 



(1) If £ V(40 < 00, then V PI U A ) = 0. 

n=l \n=l i=n / 

(2) Suppose that {A n ,n > 1} are pairwise independent with respect to v, i.e. 

' oo \ oo 



oo 
n=l 



v nu*)='- 

\n=l i=n 



Proof. 



o < v( n u a) 

n=l i=?i 
oo 

< V(UA) 

i=n 

oo 

< £ V(A) -> 0, as 7i oo. 

The proof of (1) is complete. 

oo oo 

o < i - v( n u a) 

n=l i=n 
oo 

= 1 - lim V( U A) 

oo 

= lim [1 - V( (J A)] 

oo 

= lim^(nA c ) 

oo 

= lim ]1 v{Al) 

oo 

= lim I1(1-V(A)) 

oo 

< lim ]1 exp(-V(A)) 

oo 

= lim exp(- £ V(A)) = 0. 

We complete the proof of (2). 

With the notion of IID under sub-linear expectation, Peng shows a weak law of large 
numbers for sub-linear expectation (see Theorem 5.1 in Peng [H]). 
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L/EMMA 3 (Peng [14]) Let be a sequence of IID random variables with finite 

means fx := E[Xl], ji := £[X\\. Suppose E[|Xi| 1+a ] < oo for some a > 0. Then for any 
bounded, continuous and linear growth function 0, 

— > sup as n — >■ oo. 

fJ.<X<JI 



3 The Main Results 

We now prove our main results, let us first prove the following Lemma. 

L/EMMA 4 Given sub-linear expectation E, let {Xj}^ 1 be a sequence of independent 
random variables such that sup E[|Xj| 1+Q ] < oo for some constant a > 0. Suppose that 

i>l 

there exists a constant c > such that 

\X n -E[X n ]\ <c 



n 



i /-i i V n = 1 ' 2 '"' • 
log 1 + n) 



Then there exists a sufficiently large number m > 1 such that 

mlog(l + n 



sup E 

n>l 



cxp 



-S r 



n 



< oo, 



where S n := ~ E[Xj 



PROOF. It is easy to prove that sup E[|Xj| 1+Q ] < oo implies supE[|Xj|] < oo and 



i>l 



i>l 



supE[|Xi -E[Xi 



ll+Ql 



< OO. 



i>l 



For given a > 0, noting that p og ^" w ^ 1+a — > oo as n — > oo, thus, there exist a constant 
n > and a sufficiently large number m > 1 such that 



So 



[log(l +n)] 1+a 



mlog(l + n) 



> m 1+a , > n . 



< 



1 



n > n . 



n J n 

On the other hand, it can be directly verified that the following inequality holds: 



Set x 



m log(l+n) 



e x < 1 + x+ \x\ 1+a e 2lxl , i6i, 1 > a > 0. 
[X k - E[X k }} and put it to (TJ. 



(2) 



It suffices to obtain that, for each n > k, k = 1, 2, • • • , 
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exp(^±^[X A -E[^]]) < l + ^Ml±nl[X fe -E[X fcJ 

mlog(l+n) 



+ 



\X k -E[X k ]\ 1+a e 



• 2m log(l + n) 



\X k -E[X k ] 



By assumption, it is easy to obtain 

mlog(l + n) 



n 



\X k -E[X k ]\ <cm. 



This with (JTJ, we have 
to log(l + n 



cxp 



[X k -E[X k \\ < 1 + 



TOlog(l + n) r „ . \X k -E[X k ]\ 1+a 2cm 



71 



-[X k -E[X k ]] + 



71 



Taking expectation on both sides of the above equality and setting 



we have 



E 



L:=supE[|X fc -E[X fc | 

k>l 



m logfl + n) . . 
exp ( 2i ^[X fc - E[X fc ] 



il+ai 



77 



< 1 + -e 
n 



2cm 



By the independence of {Xj}™ =1 , 



E 



cxp 



mlog(l+n) 



= nE exp(^l±!l)[x fc -E[X fc ] 

fc=i L v 
< (1 + ^ e 2cm ) n -> e Lfi2cm < oo, as n -> oo. 



The proof is complete. 



THEOREM 1 Lei {Jfi}^ be a sequence of IID random variables for sublinear expec- 
tation E. Suppose E[\Xi\ 1+a ] < oo for some a G (0, 1]. Set ~p, := E[Xl], ji = S[Xi] and 



S n := Yl x i- Then 
i=i 

(I) 



Also 



V ( {liminf S n /n < /i} I l{limsup S n /n > ji\ ] = 0. 



(3) 



v \ ji < lim inf S n j n < lim sup S n /n < ri\ — 1. 



(4) 
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(II) 



V ( limsup S n /n — /i J = 1, V (liminf S n /n = /i ) =1. 



(777,) Suppose that C({x n }) is the cluster set of a sequence of {x n } in R, then 

V(C{{S n /n}) = \ B ft) = l. 

PROOF. First, it is easy to show that argument (3) is equivalent to the conjunction 

of 

V I lim sup S n / n > Jl J = 0, (5) 

\ n—>oo J 

V (liminf S n /n < u) = 0. (6) 
In fact, writing A := {lim sup S n /n > Jl}, B := {liminf S n /n < /x}, the equivalence can 

n— >oo n— >oo 

be proved from the inequality 

max{V(A), V(B)} < V(A \J B) < V{A) + V(B). 

We only need to prove §5§ , since the proof of is similar for the reason of symmetry 
equation. 

We shall finish the proof of (|5]) by two steps. 

Step 1 Assume that there exists a constant c > such that \X n — ~fi\ < log ^ +n ^ for 
n > 1. Thus, {X i \°l l satisfies the assumptions of Lemma 4. 
To prove (jSJ), we shall show that for any e > 0, 



V [C][j{Sk/k >]I+e}) =0. 



\n=l k=n 

In fact, by Lemma 4, for e > 0, let us choose m > 1/e such that 



supE 

n>l 



exp 



ml0S( l + n) ±(X b -E\X„}) 

k=l / 



< oo. 



By Chebyshev's inequality, 

V(S n /n>Jt + e) = V(^>e) 

m log(l+ra) 
n 

< e -emlog(l+n)]g 

< (T^p^supE 



V ( mios y +n; £ - Jl) > emlog(l + n 

fe=i 

/ r> 

m log(l+n) 



n>l 



exp 
exp 



E (** - 71) 

fe=l 

n 
k=l 



(7) 
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Since em > 1, sup E 

n>l 

oo 

of E (i+n)em ) we have 

n=l 



n 

exp( nUo^ ± n) E( ^__ ) 



fc=l 



< oo. Following from the convergence 



J2 Y ( S n/n > Ji + e) < oo. 

n=l 

Using the first Borel-Cantelli Lemma, we have 

V (limsupS n /n > Ji + e) = Ve > 0, 



which implies 

V I limsup S^/n > ~p J =0, 

\ rn-oo / 

and 

v I limsup^/n < /Z J = 1- 

\ n—toc / 

Similarly, considering the sequence {— Xj}?^, by step 1, it suffices to obtain 



V (limsup(-S' n )/n > E[-X 1 ] J = 0. 



Hence, 

V fliminf Sjn < -E[-xA = 0. 

But pi = — E[— X^, hence 

V fliminf S n /n < //) = 0, 

and 

v ( lim inf S'n/n > /i )= 1 . 

Step 2. Write 

X n := (X„ - JZ) J { | X „_ P |< <» } - E (X n - /l)/ { |x n -p|< T -^ y } 



Immediately, E[X„] = Jl. Moreover, it is easy to check that {X i }°? =1 satisfies the 
assumptions in Lemma 4. 

Indeed, obviously for each n > 1, 

An - A*| < 



log(l + n) 

On the other hand, for each n > 1, it easy to check that 

|X n -/z| < |X n -zz| + E[|X n -/z|]. 
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Then 

E[|X n - < 2 l+a (E[\X n - Jc\ 1+a ] + (E[\X n - Ji\]) 1+a ) < oo. 

n 

Setting S n := E X h immediately, 

i=l 

111™ 1 11 

-S n <~Sn + -J2 \ X * - ^l 7 {l^-Pl>I3lffe)} + " E E " / J l / {I^-Ml> 1 ^)} 

i=l i=l 

Since {Xj — /l}^ have common distributions, we have 



3 



oo 

ii+Qi fn« [io g (i+i)] a 



[iog(i+»)] c 

= E[|X 1 -/l|^](i) a E ii£ ¥^ i:i <oo. 



=i 

By Kronecker Lemma, 

I 

i=i 

Furthermore, write := — ~p\ > log (/ + ^ } for i > 1. It suffices now to prove that 



1 ™ 



->■ 0. 



v(nu^)=«- 

In fact, by Chebyshev's inequality, 



v?i=l i=n 



C I 



loefl + i" 



l+a 



Hence, 



Yy( \Xi--pl > - f % \ < oo 

^ V 1 1 iog(i+o/ 



and by the first Borel-Cantelli Lemma, we have VI P| |J Ai ) = 0. 

\n=l i=n 

oo 

This implies that u> £ f] {J Ai, the sequence 

n=l i=n 



E 



i=i 

converges almost surely with respect to V as n — > oo. 
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Applying Kronecker Lemma again, 



n . 

2=1 



Setting limsup on both side of (jHJ), then by ([9]) and ( TTUi) . we have 

lim sup S n /n < lim sup S n /n } a.s. V. 

Since {I„}™ =1 satisfies the assumption of Step 1, by Step 1, 

V(lim sup S n / n > ~p) — 0, 

and also 

v (limsup S n /n < fl) — 1. 

n— >oo 

Therefore, the proof of (I) is complete. 

To prove (II), if /Z = /i, it is trivial. Suppose /Z > //, we only need to prove that there 
exists an increasing subsequence {n^} of {n} such that for any < e < /Z — //, 



oo oo 



Then we have 



vm=l fc=m 



V ( limsup S nk /n k > u. ] = 1. 

fc— ^oo 



This with (I) suffices to yield the desired result (II). 

n 

Indeed, choosing rik = k k for k > 1 and setting S n := XX^* ~V)i we have 



i=i 



^ n k -n k _ 1 — r" 



v( 




> 


. n k -n k _ 1 


v( 


' S nk -n k _ 1 - 


-(n, 


v "fc- 


-n k 


V | 




> 


l n k -n k _ 1 


E[< 


i/ Sn k~ n k-i 


-)] 


\ n k -n k _ 1 





> -e 



where 0(x) is defined by 



U' 



l_ e -(*+0 ) x > -e; 
0, x < — e. 



Considering the sequence of IID random variables {Xj — /i}^. Obviously 

E[x i -7i] = o, e[x i --p) = -(ji-n). 
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Applying Lemma 3, we have, n k — n k -\ — > oo as k — > oo and 

E[0( Sn "- n "- 1 )] _). S up 0(y) = 0(0) = 1 - e- e > 0. 
n fc — -(m-m)<j/<o 

Thus 

VV nt ^i ^^-e) >VE[0( ^Z!^ )] =00 , 

Note the fact that {5^ — S' nfc _ 1 }fe>i is a sequence of independent random variables for 
k > 1. 



Using the second Borel-Cantelli Lemma, we have 



But 



lim sup — > /I — e, a.s. V. 



Sn k ^ "S'nfc ^nk-i n k ~ n-k-1 \^nk-i\ ^fe-l 



> 



n k n k - n k -i n k n fe _i n k 

Note the fact that 



and from 



we have 



Hence, 



n k - n fc _i n fc _i 

)• 1, >■ (J, as k — > oo 

limsup5' n /n < /Z, limsup(— S n )/n < — /i, 

n— >oo n— s-oo 

lim sup (S'nl/n < max{|/l|, a.s. V. 



,. 'S'nfc . ,. Sfik ^nk-i ,. n k n-k-1 ,. I^rifc-il ,• n k-l 

hmsup > limsup lim limsup lim . 

fc^oo n k fc^oo n k — n k -\ fe^oo n k k^-oc n-k-i k ^°° n k 

We conclude that 

S 

lim sup — ^ > Jl — e, a.s. V. 

fc— s-oo ^fc 

Since e is arbitrary, we have 

V ( limsup S n Jn k >~p) =1. 

fc— >oo 
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By (I), we know V ( limsup S n /n > fi ) =0, thus 



V I lim sup S n /n — fi J = VI lim sup S n / n — fi I + V I lim sup S n /n > fi J 

\ n^oo / V rt^oo / \ n— s-oo / 



> V ( limsupS'„/n > /i ) = 1. 

\ n— s-oo 

Considering the sequence of {— X n }™ =1 , we have 

V ( limsup(- 1 S n )/n = E{-X 1 



Therefore, 

V fliminf Sjn = -E[-Xi]) = 1. 

V ra— s-oo J 

But /£ = -E[-Xi], thus 

vfliminf5„/n = /i N ) = 1. 

The proof of (II) is complete. 

To prove (III), obviously by (I), we have 

Y(C({S n /n})c[fi,Ji]) = l. 

We still need to prove 

Y(C({S n /n})D [f±,JZ}) = 1. 
If JZ = fi, it is trivial. Suppose JZ > fi, we only need to prove that, for any rational number 

6 e {ft, jz) 

Vfliminf \S n /n-b\ = o) = 1. 

V n— s-oo / 

To do so, we only need to prove that there exists an increasing subsequence {n^} of {n} 
such that for any rational number b G (fi,JZ) 



CO CO 



V fi \J{\SnJn k -b\<e})=l. (12) 



\m=l k=m 



Indeed, for any < e < min{/i — b, b — fx}, let us choose = k for k > 1. 

n 

Setting := XX-^i — then 



i=i 

V 



f| 5n * Swfc - 1 -61 < e") = vfl 5 ^"^- 1 -61 <el 

V n fc -n fc _i / V n fc-™fe-i 1 — / 



1 s »k- 


-"fc-l 


' «fc- 
s ™fc- 


■"fc-l 
~ n k-l~ 




n k ~ 




-"fc-l 


' rik- 


-Hfc-l ' 


( Sn k 


-"fc-l ^ 



— v -"fc-"fc-i < e 
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where <p(x) is defined by 



[x 



1 — e' : 
0, 



\x\ < e; 
\x\ > e. 



Considering the sequence of IID random variables {Xj — obviously 

E[Xi -6] =7* — 6 > 0, -6] =ji-b< 0. 

Applying Lemma 3, we have, as k — > oo, 



E[ 



-)] -> sup = 0(0) = 1 - e~ e > 0. 



^fc — Wfc-i L-i-b<y<Jl-b 



Thus 



OO /• Q Q \ OO Q 

Ev(i^f-n< e )>Ew^)] = <»- 



Note the fact that the sequence of {S Hk — S nkl } k >i is independent for all k > 1 . Using 
the second Borel-Cantelli Lemma, we have 



hmmt | 

fc^oo n k — n k -i 



b\ < e, a.s. V. 



But 



^-6 


< 







Q _ Q 



n k - n fc _i 



+ 



ljwfc-i I 



+ I&I 



n k -i 
n k 



(13) 



Noting the following fact, 



and 



which imply 



n k - n k -i n k _i 

> 1, > U, as A; — >• oo 

n k n k 

limsup S n /n < ~p, limsup(— S n )/n < —fi, 

n— >oo n— >oo 

limsup \S n \/n < max{|/Z|, |/x|} < oo. 



Hence, from inequality ffTBj) . for any e > 



liminf | — - — 6| < e, a.s. V. 



k^-oo n k 



That is 



V (liminf \S n /n — b\ 

\ n— >oo 



<e =1. 



Since e is arbitrary, we have 



V (liminf \S n /n-b\ =o) =1. 

V n— >oo / 



The proof of (III) is complete. 
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